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Abstract 

Message-passing algorithms based on Belief-Propagation (BP) are successfully used 
for many applications including decoding error correcting codes and solving constraint 
satisfaction and inference problems. BP-based algorithms operate over graphs represen- 
tations, called factor graphs, that are used to model the input. Although in many cases 
BP-based algorithms exhibit impressive empirical results, not much has been proved when 
the factor graphs have cycles. 

This work deals with packing and covering Linear Programs (LPs) in which the con- 
straint matrix is zero-one, the constraint vector is integral, and the variables are subject to 
box constraints. We study the performance of the Min-Sum algorithm when applied to the 
corresponding factor graph models of packing and covering LPs. 

We prove that if the LP has an optimal fractional solution, then for each fractional com- 
ponent, the value computed by the Min-Sum algorithm is either undetermined or oscillates 
between an integer less than the fraction and an integer greater than the fraction. This im- 
plies that the Min-Sum algorithm computes the optimal integral solution only if the LP has 
a unique integral optimal solution. 

Our result unifies and extends recent results for the maximum weighted matching prob- 
lem by [Sanghavi et a/.,'201 1] and [Bayati et ah, 2011] and for the maximum weight inde- 
pendent set problem [Sanghavi et al. '2009]. 

1 Introduction 

Message passing algorithms based on Belief -Propagation (BP) have been invented multiple 
times in various variants (see HGal63l IVit671 Pea88 1). Many thousands of papers report em- 
pirical results that demonstrate the usefulness of these algorithms for decoding error correct- 
ing codes, inference with noise, constraint satisfaction problems, and many other applica- 
tions HYedllH . In fact, algorithms for decoding of Turbo codes PBGT93H and LDPC codes [Gal63] 
are special variants of BP nMMC981IWib96ll 

Message passing algorithms model the problem at hand using & factor graph. The factor 
graph is a bipartite graph, one side of which consists of variable vertices, while the other side 
consists of constraint vertices. The algorithm proceeds by sending messages in rounds from 
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the variable vertices to the constraint vertices and back. In the context of linear error correcting 
codes, the factor graph simply represents the check matrix: the variable vertices correspond to 
the columns and the constraint vertices correspond to the rows. 

In this paper we focus on a special variant of BP called the Min-Sum algorithm. In essence, 
the decision of the Min-Sum algorithm in each variable node v equals the outcome of a dy- 
namic programming algorithm over a path-prefix tree rooted at v. Since dynamic programming 
computes an optimal solution over trees, the Min-Sum algorithm is optimal when the factor 
graph is a tree [Pea88 t Wib96). A major open problem is to analyze the performance of BP 
(or even the Min-Sum algorithm) when the factor graph is not a tree. This problem is often 
referred to as loopy BP. 

Recently, a few papers have studied the usefulness of the Min-Sum algorithm for solving 
optimization problems, for example: maximum weight matching [BSS08, BB CZllllSMWllll . 
maximum weighted independent set [SSW09], and min-cost flow [GS W12H . The results in 
this paper extend and generalize some of these results via a unified framework of packing and 
covering linear programs (LP). 

A packing LP is the optimization problem argmax{w T ■ x \ A ■ x < b, < x < X}, 
where w is a nonnegative vector of weights, the constraint matrix A is zero-one, the constraint 
vector b as well as the range vector X are integral and nonnegative. Similarly, a covering LP 
is the optimization problem argmin{-u; T • z \ A • z > b, < x < X}. Packing and covering 
LP's are fractional relaxations of many important optimization problems, including: maximum 
independent set, minimum weight vertex cover, and minimum weight set cover. One may 
attempt to solve integral optimization problems by solving the corresponding LP relaxation; 
success is declared if the solver returns an optimal solution that happens to be integral. (In fact, 
LP-decoding for error correcting codes is based exactly on this method [FW K05H .) 

Our main contribution is a proof that the Min-Sum algorithm is not better than linear pro- 
gramming for solving packing and covering LP's (Theorem |4] Corollary [5). More precisely, if 
there exists an optimal solution x* for the LP relaxation such that x* is not an integer, then the 
Min-Sum either declares a failure for the variable v or the integer value computed for v oscil- 
lates below and above x*. This result has a few implications: 1. Suppose the extreme points 
of the LP relaxation are integral (and hence linear programming solves the problem). If the LP 
has multiple optimal solutions, then it must have a fractional one, and the Min-Sum algorithm 
fails. 2. A fractional LP solution implies also that the Min-Sum algorithm does not converge. 
Even increasing the number of iterations ad infinitum does not help. 3. If the Min-Sum algo- 
rithm outputs the same decisions in two consecutive iterations, then this integral solution is the 
unique optimal solution of the LP. 

The converse statement does not hold in general. Sanghavi et al. [SSW09] presented a 
maximum weight independent set instance in which the LP relaxation has a unique optimal 
integral solution whereas the Min-Sum algorithm oscillates. However, the converse has been 
shown to be true for packing problems with zero-one variables by [SMW1 1 J provided that each 
variable appears in at most two constraints. We extend this result to arbitrary box constraints 
(see Appendix |B]). 

Our proof is based on the following techniques. The result of Ruozzi and Tatikonda HRT12B 
interprets rational solutions of the LP as projections of integral solutions over a graph cover. 
Since the Min-Sum algorithm operates over a path-prefix tree of the factor graph, the same exe- 
cution takes place over any graph cover. The proof proceeds by creating "hybrid" solutions that 
either refute the optimality of the LP solution or the optimality of the dynamic programming 
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solution over the graph cover. 



2 Preliminaries 

2.1 Graph Terminology and Algebraic Notation 

Algebraic Notation. Let x E R n . Denote the i\ norm of x by \\x\\ x = J2i \ x i\- The cardinal- 
ity of a set S is denoted by l^. We denote by [n] the set {0, 1, 2, n} for n e ]N + . For a set 
5 C [n], we denote by E R.' 5 ' the projection of the vector x onto indices in S. 

A vector is rational if all its components are rational. Similarly, a vector is integral if all 
its components are integral. A vector is fractional if at least one of its components is not an 
integer. 

Let X E W 1 denote a non-negative integral vector. Denote the Cartesian products of \X\] x 
• • • x [X n ] by ZBox(A'). Similarly, denote the Cartesian products of [0, X^ x • • • [0, X n ] by 
RBox(A'). Note that vectors in RBox(X) are real whereas vectors in ZBox are integral. 

Graph Terminology. Let G = (V, E) denote an undirected simple graph. Let Ng{v) denote 
the set of neighbors of vertex v E V (not including v itself). Let deg G (v) denote the edge degree 
of vertex v in a graph G, i.e., deg G (t> ) = \Af G (v)\. For a set S C V let Af G (S) = \J v< z S -^'g(v). 
A path p = (v, . . . , u) in G is a sequence of vertices such that there exists an edge between every 
two consecutive vertices in the sequence p. Let \p\ denote the length of a path p, i.e., the number 
of edges in p. Let dc(r, v) denote the distance (i.e., length of a shortest path) between vertex r 
and v in G, and let girth(G) denote the length of the shortest cycle in G. Let B G (v, t) denote 
the set of vertices in G with distance at most t from v, i.e., B G (v , t) = {u E V | d G (v , u) < t}. 

An induced subgraph is a subgraph obtained by deleting a set of vertices. The subgraph of 
G induced by S C V consists of S and all edges in E, both endpoints of which are contained 
in S. Let Gs denote the subgraph of G induced by S. A subset S C V is an independent set if 
there are no edges in the induced subgraph Gs- 

A graph G = (V, E) is bipartite if V is the union of two disjoint independent sets. We 
denote by G = (V U C, E) a bipartite graph if V fl C = 0, and V and C are independent sets in 
G. 

2.2 Covering and Packing Linear Programs 

We consider two types of linear programs called covering and packing problems. In both cases 

the matrices are zero-one matrices and the constraints vectors are positive. 

In the sequel we refer to the constraints x E ZBox(X) and x E RBox(A') as box constraints. 

Definition 1. Let A E {0, l} mxri denote a zero-one matrix with m rows and n columns. Let 
b E R.™ denote a constraint vector, let w E R n denote a weight vector, and let X E IN™ denote 
a domain boundary vector. 

[PIP] The integer program argmax {w T ■ x | A • x < b, x E ZBox(A')} is called a packing 
IP, and denoted by PIP. 

[CIP] The integer program argmin {ty T • x | A • x > b, x E ZBox(A')} is called a covering 
IP, and denoted by CIP. 
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[PLP] The linear program arg max { 
LP, and denoted by PLP. 

[CLP] The linear program arg min { 



w T ■ x | A ■ x > b, x G RBox(A')} is called a covering 



w T ■ x | A ■ x < b, x G RBox(A')} i5 called a packing 



LP, and denoted by CLP. 

2.3 Factor Graph Representation of Covering and Packing LPs 

The Belief-Propagation Algorithm and its variant called the Min-Sum Algorithm deal with 
graphical models known as factor graphs (see e.g. [KFL01]). In this section we define the 
factor graphs that are used to model covering and packing problems. 

Definition 2 (factor graph model of packing problems). A factor graph model for PIP and 
PLP, denoted by a quadruplet (G, $, X), consists of the following components: 

• A bipartite graph G = (V U C,E) that represents the zero-one matrix A. The set of 
variable vertices V = {vi, . . . , v n } corresponds to the columns of A, and the set of 
constraint vertices C = {Ci, . . . , C m } corresponds to the rows of A. The edge set is 
defined by E = {(t>j, Cj) \ Aji = 1}. 

• The vector X G IN™ defines the alphabets that are associated with the variable vertices. 
The alphabet associated with v equals [0, X v ] (resp. {0, . . . , X v }) in the case of a PLP 
(resp. PIP). 

• A collection of local (constraint indicator) factor functions^ = {?pc '■ RBox(A' J v' G ((7)) — > 
{0, — oo}} cec, where the local factor function tjjc that is associated with the constraint 
vertex C G C is defined by 



• A collection of variable functions $ = {(p v : [0,^] — > R}„ e v> where the variable 
function <p v is associated with the variable vertex v G V, and defined by (j> v ((3) = w v ■ (3. 

A word x G R ra is an assignment to variable vertices in V where Xi is assigned to vertex 
An assignment x is valid if it satisfies all the constraints (including the box constraints). 

The factor graph model allows for the following equivalent formulation of the packing 
integer program: 




(1) 



arg max { ^ <t>vM +^2^c{xm g (C)) \ G V. x v G 



{0,...,^}}. 



(2) 



Analogously, the packing linear program PLP is equivalent to 



arg max { ^ <p v (xv) + ^c{xu g {q) \ G V. x v G 
vev cec 



[0,X V ]}. 



(3) 
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We may define a factor graph model for covering problems in the same manner. The only 
difference is in the definition of the collection of factor functions \& = {ipc '■ RBox(A' A r G (c)) — > 
{0, oo}} c&c defined by 

covering factor: ipc\V) = S Gl w (4) 

I otherwise 

Using this factor model, we can reformulate the covering integer program CIP by 

argmin { S^vjxy) + ? J 1 Pc{xjsr G (c)) | E V. x v E {0, . . . , X v }}. (5) 

uev cec 

The covering linear program CLP is equivalent to 

argmin { ^4> v (x v ) + ^^c(^ G (c)) | E V. x v E [0, X v ]}. (6) 

v&v c&c 

3 Min-Sum Algorithms for 0-1 Packing and Covering Inte- 
ger Programs 

In this section we present the Min-Sum Algorithm for solving packing and covering integer 
programs with zero-one constraint matrices. Although some representations of the algorithms 
presented in this section actually perform a max-sum operations rather then min-sum opera- 
tions, we refer to these algorithms in general as min-sum algorithms. All the results in this 
section apply to any other equivalent algorithmic representation (e.g., max-product-type for- 
mulations). We first define the min-sum algorithms for PIPs and CIPs, and then state our main 
results. 

3.1 The Min-Sum Algorithm 

The Min-Sum Algorithm for packing integer program (PIP) is listed as Algorithm \T\ The 
input to algorithm min-sum-packing consists of a factor graph model (G, ^, $, X) of a PIP 
instance and a number of iterations N E IN. Each iteration consists of two parts. In the first part, 
each variable vertex performs a local computation and sends messages to all its neighboring 
constraint vertices. In the second part, each constraint vertex performs a local computation and 
sends messages to all its neighboring variable vertices. Hence, in each iteration, two messages 
are sent along each edge. 

Let $\q{P) denote the message sent from a variable vertex v E V to an adjacent constraint 
vertex C E C in iteration t under the assumption that vertex v is assigned the value f3 E 
{0, . . . , X v }. Similarly, let ^§^ v denote the message sent from C E C to v E V in iteration t 
assuming that vertex v is assigned the value (5 E {0, . . . , X v }. Denote by (i v (0) the final value 
computed by variable vertex v E V for assignment of (3 E {0, . . . , X v }. 

First the algorithm computes the initial messages. These messages (considered as the zeroth 
iteration) have the value zero and are send along all the edges from the constraint vertices to 
the variable vertices. In practice, these messages are not sent, and are part of the initialization 
by the variables vertices. 
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The algorithm proceeds with N iterations. In Line l2al the message to be sent from v to C 
is computed by adding the previous incoming messages to v (not including the message from 
C) and adding to it In Line |2E1 the message to be sent from C back to v is computed. 

The constraint node C considers all the possible valid assignments y to the neighbors in which 
y v = (3. Among these assignments, the message is chosen to be the one that maximizes il>c(y) 
plus the sum of the previous incoming messages (not including the message from v). 

Finally, in Line |3] a local computation takes place in each variable vertex. Each variable 
vertex v chooses a value f3 E {0, . . . , X v } for which the sum of the last incoming messages is 
maximized. If this maximum is not attained by a unique value, then the variable vertex declares 
failure. We use the convention of using a question mark to denote the event that the min-sum 
algorithm fails to decide on a value of a variable. 



Algorithm 1 min-sum-packing((G, $, X), N) - A min-sum algorithm for a PIP 
argmax{zf T ■ x \ A ■ x < b, x E ZBox(X)}. Given the factor graph model (G, ^, X) 
of the PIP and the number of iterations N E IN, outputs a string x of length n such that 



2. Iterate: For t — ltoN do 

(a) For each (v, C) E E and f3 E {0, . . . , X v } do {variable-to-constraint message} 



Sn E {0, . . . , Ai} U {?}. 



1. Initialize: For each (v, C) E E and /? E {0, . . . , X v } do 



C'£Af(v)\{C} 



(b) For each (v, C) E E and (5 E {0, 



. . . , X v } do {constraint-to-variable message} 



max 

y:AT(C)-^ZBox(AV (c) ) s.t. y v =/3 



u£AT(C)\{v} 



3. Decide: For each v E V do 



(a) For each f3 E {0, . 



X v } do 



C&Af(v) 



(b) 




if j3 = argmax{/ii,(o;) | a E {0, . . . , X v }} is unique 
otherwise 



Return x 
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Algorithm min-SUM-COVERING listed as Algorithm |2] is based on the following reduction 
of the covering LP to a packing LP as follows. 

Claim 3. Let d = A ■ X - b E R m , then 

arg min { w T ■ z \A-z>b, ze RBox(X) } = arg max { w T ■ x \A-x<d, xe RBox(^)} 

Proof. Let x = X - z. Then, z E RBox(X) <^> x E RBox(X), and A - z > b o A - x < 
A- X -b. Then 

mm{w T ■ z | A ■ z > b, z E RBox(X)} = 

= min {w T ■ (X - x) \ A- x < A - X - b, x E RBox(X)} 

= w T -X + min {-w T -x\A-x<A-X-b, x E RBox(X) } 

= w T ■ X + max {w T • x \ A- x < A- X -b, x E RBox(A') } , 

and the claim follows. □ 

An equivalent min-sum formulation of MIN- SUM- COVERING algorithm is listed in Algorithm|3] 

Algorithm 2 min-sum-covering((G, ^, $, X), N) - a min-sum algorithm for a CIP 
argmin{w T • z \ A ■ z > b, z E ZBox(A')}. Given the factor graph model (G, $, X) 
of the CIP and the number of iterations N E IN, outputs a string z of length n such that 

i i e{o,..., < y i }u{?}. 

1 . Define a factor graph model for a PLP as follows: 

(a) Define d E W n : For each C E C do 

vGAf{C) 

(b) Let (G, $, X) denote the factor graph model for the PLP 

arg max {w T ■ x | A ■ x < d, x E ZBox(X)} 

2. Let x denote the decision of min-sum-packing((C7, $, X),N) 

3. Decision: For each v E V do 

f? ifx„=? 
z v ^— < 

I X v — x v otherwise 

Return z 



3.2 Summary of Results 

In this section we summarize the main results of this paper. The proof of Theorem |4] appears in 
Section \5\ after the presentation of the main tools of the proof. 
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Algorithm 3 min-SUM-COVERING((G, ^, $, X),N) - an equivalent min-sum formulation of 
Algorithm |2l 

1. Initialize: For each (v, C) G E and f3 G {0, . . . , A^} do 

2. Iterate: For t = 1 to N do 

(a) For each (v, C) <E E and /? G {0, . . . , do {variable-to-constraint message} 

C*'eAA(u)\{C} 

(b) For each (v, C) e E and /3 G {0, . . . , X v } do {constraint-to-variable message} 

y:A^(C)->ZBox(AV {c) ) s.t j/„=/3 ^— * 



uGA^(C)\{«} 



3. Decide: For each v e V do 

(a) For each /3 G {0, . . . , X v } do 



E 

(b) 

/3 if /3 = argmin{/^(a) | a G {0, . . . , A^,}} is unique 
? otherwise 

Return 5 
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Theorem 4 (weak oscillation of Min-Sum). Let x* denote an optimal solution of a packing LP 
arg max { w T ■ x \ A - x <b,x E RBox( X) }, and let denote the output of Algorithm min- 
SUM-packing((G, $, X), t) for the corresponding factor graph model after t iterations. 
Then, 

1. Ifxf^ 7^? and t is even, then x* < xf . 

2. Ifxf t^? and t is odd, then x* > xf . 

Proof. See Section 15^21 □ 

Theorem 0] states a necessary condition for the convergence of the MIN-SUM-PACKING al- 
gorithm to the unique integral optimal solution. Indeed, unless the PLP has a unique optimal 
solution that is integral, the min-SUM-packing algorithm will not converge to an integral so- 
lution. Specifically, for every fractional component x*, the decision of the MIN-SUM-PACKING 
algorithm oscillates between {0, . . . , [x*\ } U {?} and { \x*~\ , . . . , Xi] U {?} in every other iter- 
ation. Note that if the PLP has two optimal solutions that are integral, then their average is also 
optimal but not integral. In this case, the min-SUM-packing algorithm is bound to fail. 

The following corollary stated for covering LPs follows from Theorem[4]and Claim[3j 

Corollary 5. Let z* denote an optimal solution of a covering LP argmin{u> T -z | A-z>b, z G 
RBox(A')}, and let denote the output of Algorithm min-SUM-COVERING((G, \l>, $, X) , t) 
for the corresponding factor graph model after t iterations. Then, 

1. Ifzf* 7^? and t is odd, then z* < zf\ 

2. If zf^ 7^? and t is even, then z* > zf\ 

The converse of Theorem @] and Corollary |5]is not true in general. However, a converse 
has been shown to be true for packing problems with zero-one variables by [SMW1 1] provided 
that each variable appears in at most two constraints. We extend this result to arbitrary box 
constraints (see Appendix |B]). 



4 Graph Liftings 

The purpose of this section is to show that there exists a universal LP graph covering of arbitrary 
girth (see Corollary [12]). We present definitions and notation (as used in [AL02J), and adapt 
them to assignments and liftings of factor graphs models. We then state the main combinatorial 
characterization based on HRT12L and show how the girth can be arbitrarily increased. 

4.1 Covering maps and Liftings of Graphs and Factor Graph Models 

Definition 6 (covering^ map HAL0210 . Let G = (V, E) and G = (V, E) denote finite graphs. 
A homomorphism n : G — >■ G is a covering map if for every v G V the restriction of tv to 
neighbors ofv is a bijection to the neighbors ofir(v). 

1 The term covering is used both for optimization problems called covering problems and for topological 
mappings called covering maps. 
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fiber - ir 1 (v i) 7T 1 (t>2 ) 




^3 V4 tt" 1 ^) TT -1 ^) 

(a) Base graph (b) An M-lift of G. 

G. 



Figure 1: M-lift of base graph G: Fiber 7r x {vj) contains M copies of Vi. A matching between 
fibers 7r _1 (vj) and 7r — if (i> i3 Uj) is an edge in G. 

We refer only to finite covering maps. The pre-image -k~ 1 (v) of a vertex v is called a. fiber. 
It is easy to see that all the fibers have the same cardinality if G is connected. This common 
cardinality is called the degree ox fold number of the covering map. If n : G — > G is a covering 
map, we call G the base graph and G a lift of G (see Figure [T]). In the case where the fold 
number of the covering map is M, we say that G is an M -/(/? of G. 

If G is connected, then every M-lift of G is isomorphic to an M-lift that is constructed as 
follows. The vertex set is simply V = V x [M] and covering map is tt((v, i)) = v for every 
(v, i) G V. The edges in E are obtained by specifying a matching of M edges between 7r _1 (w) 
and 7r _1 (f) for every (u, v) E E. 

The notion of M-lifts in graphs is extended to M-lifts of factor graph models in a natural 
manner. Each variable node v inherits the variable function of tt(v). Similarly, each constraint 
variable C inherits the factor function of n(C). For brevity, we refer to the lifted factor graph 
model (G, * , $, X) of (G, $, Af) simply as the lift G of a factor graph G. 

An assignment x to the variable vertices V of a factor graph G is extended to the M-lift 
G simply by defining x^ M (v) = x(ir(v)). Note that this extension preserves the validity of 
assignments. 

One may project an assignment x of the M-lifted graph to the base graph. The projected 
assignment p(x) is defined by 

\p{x)](v) = — - (7) 

CG7T -1 (v) 

By linearity it follows that projection preserves the validity of assignments. 

4.2 Relation to PLPs and CLPs 

Let fcm(G) denote the factor graph model (G, $, X). Let <2(fcm(G)) denote the set of all 
vectors x such that there exists a fold number M and an M-lift G of G such that x is a valid 
integral assignment for G. Denote by Q(fcm(G)) the projection of the set Q(fcm(G)) to the 
base model, namely, 

Q(fcm(G)) = {p(x) | x e Q(fcm(G))} (8) 

The following theorem states that the projection of valid assignments of lifted graphs equals 
the set of all rational vectors in the polytope of the corresponding PLP/CLP. 
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Theorem 7 (special case of [RT12, Theorem VII. 2]). Let V denote the polytope of feasible 
solutions of a PLP/CLP, and let fcm(G) denote the corresponding factor graph model of the 
PLP/CLP. Then, 

Q(fcm(G)) = V n Q n . (9) 

Note that all the basic feasible solutions of the polytope V are rational and hence in the set 
<2(FCM(G)). 

4.3 Universal LP Covering 

We use the following notation. Let BFS('P) denote the set of all basic feasible solutions of 
a polytope V. Let V denote a polyhedron of a PLP/CLP with a corresponding factor graph 
model fcm(G) = (G, *, $, X). 

In this section we prove that for every g e IN there exists a finite lift G of the base factor 
graph model such that (i) BFS('P) C {p(x) \ x is a valid binary assignment for G], and 
(ii) girth(G) > g. The proof proceeds in two steps. We first obtain a finite lift G' of G that 
satisfies property (i) (see Proposition [8] and Corollary |9). Then we obtain a lift G of G 1 that 
satisfies property (ii) (see Proposition [10] and Corollary ITTI). 

The following proposition is implied from the general result proved in HLei82L A simpler 
proof for the specific case stated in the proposition is provided in Appendix lAl 

Propositions. Consider a factor graph model (G, $, X). Let (Gi, ^i, &i,Xi) and (G 2 , ^2, $ 
denote two lifted factor graph models of degree Mi and M% respectively. Then, there exists a 
lifted factor graph model (G, $, X) of degree Mi ■ M 2 such that G is both an M 2 -lift of Gi 
and an Mi- lift of G 2 . 

The following corollary proves the existence of a finite "universal" lifted integral factor 
graph model that captures the fractional optimization problem modeled by the base factor 
graph. 

Corollary 9. There exists a finite lift G of G such that 

BFS(P) C {p(x) I X IS cL valid integral assignment for G}. 

Proof. By Theorem [Vj for every basic feasible solution xofV there exists a finite lift G of G 
with an integral assignment x such that p(x) = x. By applying Proposition [8l we can combine 
any finite number of finite lifts into one "joint" finite lift. Because the number of basic feasible 
solutions is finite, the corollary follows. □ 

The following proposition deals with obtaining lifts with large girth. 

Proposition 10. There exists a finite lift G of G such that girth(G) > 2 • girth(G). 

Proof. Given a graph G = (V, E), we construct a 2 |B| -lift G = (V, E) as follows. Let k = \E\. 
The vertices in each fiber of G are indexed by a binary string of length k. Index the edges in E 
by {ei, . . . , efe}. For an edge = (u, v), the matching between the fiber of u and the fiber of v 
is induced simply by flipping the i'th bit in the index. Namely, «(6 1 ...6 i ...6 fc ) v, bl ^ b k )- 

Consider a cycle 7 in G and its projection 7 in G. Each edge ei in 7 must appear an even 
number of times. Otherwise, the z'th bit is flipped an odd number of times in 7, and 7 can not 
be a cycle. It follows that girth(G) > 2 • girth(G). □ 
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By applying Proposition [TO] repeatedly, we have the following corollary. 

Corollary 11. Consider a graph G. Then for any finite £ E IN there exists a finite lift G of G 
such that girth (G) > 2 e . 

Following Corollary |9]and Corollary [TT] we obtain the following corollary for the existence 
of a finite universal LP cover of any poly tope V with arbitrary large girth. 

Corollary 12. Let V denote a polyhedron of a PLP/CLP with a corresponding factor graph 
model (G, ^, $, X). For every g E IN, there exists a finite lift (G, X) such that: (i) BFS(P) C 
{p(x) | x is a valid integral assignment for G}, and (ii) girth(G) > g. 



5 Proof of Main Results 



5.1 Min-Sum as a Dynamic Programming on Computation Trees 

Given a graph G = (V, E) and a vertex r E V . The path-prefix tree of height h is defined as 
follows. 

Definition 13 (Path-Prefix Tree). Let V denote the set of all non-backtracking path$ with 
length at most h that start at vertex r. Let E = { (pi, p 2 ) E V x V | p\ is a prefix of p 2 , |pi | + 
1 = |p 2 | }• Wfc denote the zero-length path in V by (r). The directed graph (V, E) is called the 
path-prefix tree of G rooted at vertex r with height h, and is denoted by T r h (G). 

The graph l~ r h (G) is obviously acyclic and is an out-tree rooted at (r). Path-prefix trees of G 
that are rooted in variable vertices are often called computation trees of G or unwrapped trees 
ofG. 

We use the following notation. Vertices in T r h (G) are paths in G, and are denoted by p and 
q whereas vertices in G are denoted by u, v, r. For a path p E V, let t(p) denote the last vertex 
(target) of path p. 

For a factor graph G = (V U C, E), let V denote the set of paths in V that end in a variable 
vertex, i.e., V = {p \ p E V, t(p) E V}. Let C denote the set of paths in V that end in a 
constraint vertex, i.e., C = {q \ q E V, t(q) E C}. Paths in V are called variable paths, and 
paths in C are called constraint paths. We attach a variable functions P and factor functions 
ip p to the vertices of T r h (G); each vertex p inherits the function of t(p). The box constraint for 
a variable path p is defined by X p = X t u)- 

In the following lemma, the MIN-SUM-PACKING algorithm is interpreted as a dynamic 
programming algorithm over the path-prefix trees (see e.g., [GSW12, Section 2]). 

Lemma 14. Consider an execution of min-SUM-packing((G, $, X), t). Consider the 
computation tree 7^ t (G) = (V U C, E). For every variable vertex r EV and (3 E {0, . . . , X r }, 

Hr(f$) = max i ^ 4>v(z P ) + ^^q{zN T {q)) V P G V-Zp e ZBox(A'p), z {r) = (5 \. 



2 A backtrack in a path is a subpath that is a loop consisting of two edges traversed in opposite directions. A 
path is backtrackless if it does not contain a backtrack. 



12 



By Linel3blin Algorithm min-SUM-packing we conclude the following corollary. 

Corollary 15. Algorithm MIN-SUM-PACKING outputs x r = (3 if and only if the following 
condition holds: Every valid assignment z that maximizes the objective function X] pG v ^pi^p) + 
E g ec 4(%r(?)) f° r V\ G ) = (V U C, E) satisfies z {r) = (3. 

5.2 Divergence of min-SUM-packing - Proof of Theorem H 

Proof of Theorem\4\ We begin by assuming that x* is a basic feasible solution of the LP. 
Following Corollary [121 let G = (V U C,E) denote a universal LP cover of G such that 
girth(G) > At, where t is the number of iterations performed by the min-SUM-packing algo- 
rithm. Because x* is a basic feasible solution of the LP, there exists an integral valid assignment 
x* in G such that the projection of x* equals x*, i.e., p(x*) = x*. 

We first prove part 1 of the theorem, namely, if x^ ^ ? and t is even, then x$ > x*. 
Because x* v equals the average of x* over the fiber of v, it follows that there exists a variable 
vertex v in the fiber of v such that x* < x%. Note that x% is an integer whereas x* may be a 
fraction. We shall prove a slightly stronger claim that Xv > x% (provided that x^ 1 ^?). 

Assume for the sake of contradiction that the min-sum-packing algorithm outputs x^ = 
5 where 5 7^? and 5 < x~. 

Let Bq(v, 2t) = {u E V U C \ d(v, u) < 2t] denote the ball of radius 2t centered at v. 
Denote by G B {v,2t) the subgraph of G induced by B^(v, 2t). Because girth(G) > At, G B (v,2t) 
is a tree. Moreover, GB(v,2t) is isomorphic to the computation tree 7^ t (G). 

By Corollary [151 every optimal valid integral assignment x for 7^ 2t (G) assigns the value 5 
to the root. Because Gs(v,2t) is isomorphic to T^ t (G), the same holds for GW^t) as well. Let 
z denote such an optimal integral valid assignment to variable nodes in Gscv,2t)- 

Let £ = {u e B d (v, 2t)nV\xl> z^} and let O = {u e B d (v, 2t) n V | x\ < z a }. Note 
that by the assumption x~ > 5 = z, d , and therefore v E £. 

Let T denote the subtree of G B (v,2t) rooted at v that is maximal under inclusion subject 
to the following conditions: (i) The variables vertices in T are contained in E U O. (ii) Each 
variable vertex u e 8 is in T only if | • d(v, u) is even, (iii) Each variable vertex u E O is in 
T only if \ ■ d(v, u) is odd. (iv) A constraint vertex C is in T only if it is an interior vertex in a 



path from v to a variable vertex u in T. See Figure |2(a)1 for an illustration. We refer to T as an 
alternating tree because its variable vertex layers alternate between vertices in £ and vertices in 
O. We refer to layers that contain vertices from £ as even layers (i.e., layers 0,4,8,...). Layers 
that contain vertices from O are referred to as odd layers (i.e., layers 2, 6, 10, . . .). 

A skinny tree is a subtree of T such that each constraint vertex chooses only one child, 
and hence called skinny. Formally, a skinny subtree Ts of T is a subtree rooted at v such that 
(i) deg Ts (C) = 2 for every constraint vertex C in 7s, and (ii) deg Ts (£t) = deg T (u) for every 
variable vertex u in Ts- 

To summarize, we have used the assumption to define an alternating tree T that is rooted 
at v. We choose any skinny subtree Ts of T to contradict the optimality of z over GB(v,2t) as 
follows. Let S £ = {u E T s | u E £}, let S a = {u E T s | u E O}, and let S = S £ U S a . 
Namely, contains variable vertices of the skinny tree in even layers, and So contains variable 
nodes of the skinny tree in odd layers. By the definition of £, x is greater than z over vertices 
in §£. Similarly, by the definition of O, x is less than z over vertices in S . 
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(a) An illustration of induced subgraph (b) The alternating tree T rooted at v. Variable 

GB(v,2t) for t = 2 with sets £ and O. Nodes vertices in T are contained in £ U O. Ignoring 

in £ are colored (or outlined) in blue, and constraint vertices, paths from the root v alter- 

vertices in O are colored (or outlined) in red. nate between £ and O. 




(c) A skinny subtree 7s of the alternating tree (d) Valid assignment a for G illustrated with 

T. The root (layer 0) and vertices in layer 4 are respect to assignment x* in proof of Claim [T6l 
contained in Sg, and the vertices in layer 2 are 
contained in So- 



Figure 2: Illustrations of graphs, trees and sets in proofs of Theorem |4] and Claim[[6] Variable 
vertices are illustrated by circles and constraint vertices are illustrated by squares. 
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Assign each node u in the fiber of u a weight w u = w u . For a subset of variables vertices 
A C V, let = ^] Sg j; Wg. We rely on the following claim (which we prove later). 

Claim 16. w(S £ ) - w(S ) > 0. 

We now define an assignment y to variable nodes in G B {v,2t) by 

{z u + 1 Hue Ss 
zu-l Hue So (10) 
5u otherwise 

We show that y is a valid integral assignment for the factor graph model of G B (v,2t)- First notice 
that y satisfies the box constraint for every variable vertex u, i.e., y& G {0, . . . , Af fi }. If u G 
then y fi = lu + 1 < x* u < A^, and if u G <§e>, then ?/„ = 5„ — 1 > x u > 0. 

We now show that y satisfies every constraint vertex C in Bq(v, 2t). Consider a constraint 
vertex C in S^({;,2t). If Mq(C)(1S = 0, then the assignment to the neighborhood of constraint 
C is identical to z. Therefore, C is satisfied by y by the validity of assignment z. If J\[q(C) fl 
5^0 the we have one of the following three cases, (i) J\[q(C) fl So = {u}. In that case 
Vu = Zu — 1 while the other neighbors of C are assigned according to z, and therefore C stays 
satisfied by the validity of assignment z. (ii) Nq(C) n§£ = {u}. In that case iju = z u + 1 < . 
Moreover, every other neighbor f G Nq(C) \ {u} is not in O, and hence satisfies 5 ? < x}.. 
Hence, y satisfies C by the validity of assignment x*. (iii) Mq{C) fl S = {u, r} where without 
loss of generality u E Ss and f G So- The assignment of y to neighbors of C is obtained by 
z where the assignment to u is reduced by 1 while the assignment to f is incremented by 1. 
Hence y satisfies constraint C by validity of the assignment z. We conclude that y is a valid 
integral assignment. 

By the definition of y and by Claim [16] we have 

Mit*)- J2 MSa) = w(Se)-w{So)>0. (11) 

ueVC\B 6 {v,2t) u£VnB e (v,2t) 

Because the root v G it holds that y, d = z^, + 1 = 5+1. Thus we obtained a contradiction 
to the assumption that every optimal valid integral assignment for Gs{v,2t) assigns the value 5 
to the root v. 

We prove part 2 of the theorem stating that if Xv 7^? and t is odd, then x^ < x* using the 
following modifications. The variable vertex v is chosen so that x~ < x*. Assume for the sake 
of contradiction that Xv > x%. In this case, 5 C = x£ > x%. In the definition of the alternating 
tree the roles of £ and O are interchanged so that the odd layers contain vertices in £ while the 
even layers contain vertices in O. Because v G O and t is odd, it follows that each leaf u of 
the skinny tree 7s in layer 2t belongs to £ , and the proof of Claim [16] remains valid. Finally, 
the assignment y assigns zr u — 1 to v, which contradicts the assumption about optimal valid 
assignments to G B (v,2t)- 

We now complete the proof for the case that x* is not a basic feasible solution. In this 
case x* is a convex combination of optimal basic feasible solutions. This implies that for every 
variable vertex v, there exist optimal basic feasible solutions y' and y n such that y' v < x* v < y" v . 
For an even t, if x^ 7^?, we have x* v < y" v < Xv\ For an odd t, if 7^?, we have 
x% > y' v > Xv \ and the theorem follows. □ 
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Proof of Claim 1761 Define an assignment a to variable vertices in V by 

{xl + 1 if fi e 5 
x\-l iiueSs 
x~ otherwise 

The proof that a is a valid assignment for G mimics the proof that y is a valid assignment in 
the proof of Theorem |4] The only new observation that is needed is that a constraint vertex 
C in G \ Gb(v,2£) may have a variable vertex in S as a neighbor (i.e., variable vertices in the 
boundary of the ball Bq(v, 2t)). However, each variable vertex at distance 2t from the root v 
belongs to S. Hence, = x\ — 1, and decrementing the value assigned to u with respect to x* 
does not violate the constraint C. 

By the optimality of x* we have that, w T ■ (x* — p(a)) > 0. Hence, w T ■ (x* — a) > 0. By 
the definition of a, we have w T ■ (x* — a) — w(Se) — w(So), and the claim follows. □ 
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A Proof of Proposition IS - Finite Common Lifted Graph 

Proof. For i G {1,2} consider the Mi-lift (G h $ i; X,) of (G, and let vr; : G< -> G 

denote the corresponding covering map. Then, for every edge (it, v) G E there is a perfect 
matching between the fibers 7r~ 1 (t>) and Tr~ x {u). Let a^' : ^^(v) — > vr i ~ 1 (tt) denote the 
matching map for every edge (u, v) G E. 

We construct an (Mi • M 2 )-lift G of G as follows (see Figure |3]for illustration). Index the 
vertices in each fiber of G by (i, j) G [Mi] x [M 2 ]. Denote by the vertices in the fiber of 
v. It is convenient to interpret each matching af" as a permutation af 1 : [Mi] — > [Mi]. The 
edge set E of G is defined as follows. 

{u(i,j),V( k j)) e E iff (u,v) G E, and 

k = a[ u,v \i), and 
I = *<?%). 

It remains to prove that G is an M 2 -lift of G\ and an Mi-lift of G 2 . Let Kq^Gi '■ V Vi 
denote a mapping from V onto V\ defined by 7Tg-><5i ~ (^)- Clearly, the mapping 
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Figure 3: Constructing an (Mi ■ M 2 )-lift G of G such that G is an M 2 -lift of G x and an M r lift 
of G 2 , in proof of Proposition [8] 

7Tg->Gi ma P s each e dge V(k,e)) G -E to an edge (ui,Vk) € -Ei, where A; = crp^(i) and 

^ = 2 (.?')• ^ a ^ so h°lds that for every w^j) e V - the restriction of Tr^g to A/g(S(ij)) is 
a bijection to A/g (u*). Hence, ttq_^q is a covering map of degree M 2 from G onto Gi. An 
analogous covering map can be defined from G onto G 2 which concludes the proof. □ 

B On Convergence of the Min-Sum Algorithm for Packing 
and Covering Problems 

In Section [372] we showed that if the MIN-SUM-PACKING algorithm (respectively, MIN-SUM- 
COVERING algorithm) converges (i.e., outputs the same decision in two consecutive iterations), 
then its output is optimal and the LP relaxation of the PIP (resp., CIP) is tight. However, the 
converse statement is not true in general. Namely, if the LP relaxation of a PIP is tight, then the 
MIN-SUM-PACKING algorithm is not necessarily guaranteed to converge to the optimal integral 
solution. Sanghavi et al. [SSW09] presented a counterexample in which the max-product (min- 
sum) algorithm does not converge to the unique integral optimal solution of a maximum weight 
independent set LP. In this section we present a generalization of the convergence result of the 
min-sum algorithm in [SMW1 1 1 to packing and covering problems represented by factor graphs 
whose maximum variable vertex degree equals two. 

Sanghavi et al. HSMW11I noticed that the minimum rate of convergence of the max-product 
algorithm for the maximum weighted matching problem depends on the polyhedron of the 
packing linear program. Loosely speaking, given a polyhedron V and a cost vector w, c(V, w) 
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is the minimum slope between an optimal basic feasible solution of a maximization problem 
and the other basic feasible solutions. 



Definition 17 ([SM WlllO . Given a polyhedron V C R n and a cost vector w G R n . Define 

c(V, w) by 



c(V, w) = min 



A . w T ■ (x* — x) 



x£V\{x*} \\X* — X\\i 

where x* = argmax{w T • x \ x G V}. 

Because x* is defined to be the optimal solution of a maximization problem, c(V, w) is non- 
negative. Note that c(V, w) — if and only if there are multiple optimal solutions to the LP 
argmax{zi; T ■ x \ x E V}. Let u> max = m&x{vji | 1 < i < n} denote the maximal component 
of the weight vector w G R n . It also holds that c(V, w) < w max . 

The following theorem generalizes the convergence result for the maximum weighted match- 
ing problem HSMW11I to any problem defined by a factor graph model such that the maximum 
variable vertex degree equals two. Namely, the MIN-SUM-PACKING algorithm converges to the 
optimal solution of a packing LP in pseudo-polynomial time, provided that the degree of the 
variable vertices in the factor graph model is at most two. 

Theorem 18. Consider a packing LP argmax{u; T ■ x \ A ■ x < b, x G ZBox(A')} such that 
every column of A contains at most two Is. Assume that the PLP has a unique optimal solution 
x* that is integral, and let (G, $) denote the corresponding PLP factor graph model. Let 

V denote the polytope {x G RBox(A') | A ■ x < b}. Let x^ denote the output of Algorithm 
MIN-SUM-PACKING((G, \[>, $), t) after t iterations. Ift > + 1 then x^ = x*. 

Prior to proving Theorem [T8] we provide the following corollary that states an analogous 
convergence result for the MIN-SUM-COVERING algorithm with respect to covering LPs. The 
corollary follows from Theorem [18] and Claim |3] 

Corollary 19. Consider a covering LP argmin{ti; T • z | A ■ z > b, z6 ZBox(A')} such that 
every column of A contains at most two Is. Assume that the CLP has a unique optimal solution 
x* that is integral, and let (G, $) denote the corresponding CLP factor graph model. Let 

V denote the polytope {z G RBoxAf | A ■ z > b}. Let denote the output of Algorithm 
MIN-SUM-COVERING((G, ^, $}, t) after t iterations. Ift > ^frz^y + 1 then = z*. 

Proof of Theorem 17^1 Let x* G ZBox(A') denote the unique optimal solution of the packing 
LP. Note that the optimal basic feasible solution of the LP is integral and unique, otherwise 
there exists a factional optimal solution. We prove that * for every variable vertex 

v G V provided that t > . 

Consider a variable vertex v. We first prove that the output of the MIN-SUM-PACKING 
algorithm cannot be greater than x*. By the same reasoning, we prove that x$ cannot be 
smaller than x*. We conclude by showing that x-tP 7^?, and hence 

Following Corollary \TT\ let G = (V U C, E) denote a lift of G such that girth(G) > At, 
where t is the number of iterations performed by the MIN-SUM-PACKING algorithm. Let x* 
denote the lift of assignment x* to G. Because x* is an integral valid assignment for G, x* is an 
integral valid assignment for G such that p(x*) = x*. Let v denote any variable vertex in fiber 
7r _1 (v). Clearly, x% = x*. 
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Assume for the sake of contradiction that the min-SUM-packing algorithm outputs x v = 
5 where 5 7^? and 8 < x~. 

Let Bq{v, 2t) = {u G V U C \ d(v, u) < 2t} denote the ball of radius 2t centered at v. 
Denote by G B {v,2t) the subgraph of G induced by Bq{v, 2t). Because girth(G) > At, G B (v,2t) 
is a tree. Moreover, G B (v,2t) is isomorphic to the computation tree T v 2t (G). 

By Corollary \\5\ every optimal valid integral assignment x for 7 v 2t (G) assigns the value 5 
to the root. Because G B ( v ,2t) is isomorphic to T v 2t (G), the same holds for G B (v,2t) as well. Let 
z denote such an optimal integral valid assignment to variable nodes in G B (v,2t)- 

The rest of the proof that x$ x* v mimics the proof of Theorem |4] (Part 1) as follows. We 
define the sets 8 and O, the maximal alternating tree T, skinny tree 7s and the sets Sg, So and S 
as in the proof of Theorem |4] The key difference is in the statement and proof of the upcoming 
Claim [20] (which we prove later) that now incorporates the condition that t > ^r^y + L 

Claim 20. Ift > + 1, then w(S £ ) - w(S ) > 0. 

We then define a valid integral assignment y for the factor graph model Gb(v,2£) as in Equa- 
tion (flOl) in the proof of Theorem |4] The assignment y satisfies 

^2 faiVu) > ^(zu), (12) 

ueVr\B 6 (v,2t) nGVnB 6 (C,2t) 

Because the root v 6 Sg, it holds that y^, = + 1 = 5+1. Thus we obtained a contradiction 
to the assumption that every optimal valid integral assignment for G B {v,2t) assigns the value 5 
to the root v. 

We prove that Xv cannot be greater than x* using the following modifications. We choose 
any variable vertex v G 7r _1 (v). It holds that * Assume for the sake of contradiction that 

Xv > x*-. In this case z^ = xi > %%■ In the definition of the alternating tree the roles of 8 and 
O are interchanged so that the odd layers contain vertices in 8 while the even layers contain 
vertices in O, and the proof of Claim l20~lremains valid. Finally the assignment y assigns Zy — 1 
to v, which contradicts the assumption about optimal valid assignments to G B ( v ^t)- 

To conclude the proof we need to prove that xi^ 7^? for t > ^rgy + 1- Assume for 

the sake of contradiction that x$ =?. In this case, there exists at least two optimal valid 
integral assignments for the factor computation tree model in which the root is assigned with 
two different values. At least one of these values is either greater than x* (a case that reduces to 
the first part of the proof), or less than x* (a case that reduces to the second part of the proof). 
In either of these cases, we get contradiction to the assumption that Xv =? which concludes 
the proof. □ 

Proof of Claim \2U\ Define an assignment a to variable vertices in V by 

if u G So and d(v, u) < 2t 
if u G Sg 
otherwise 

The proof that a is a valid assignment mimics the proof that y is a valid assignment in the 
proof of Theorem |4] The only new observation that is needed is that a constraint vertex C in 
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G\GrB(v,2t) ma y have a variable vertex in S as a neighbor (i.e., variable vertices in the boundary 
of the ball B^v, 2t) that are contained in S). However, the distance of such a neighbor u from 
the root v equals 2t. Therefore, if u belongs to E then a s = x\ — 1, and decrementing the value 
assigned to u with respect to x* does not violate the constraint C. Otherwise, *- which 

leaves constraint C satisfied as with respect to assignment x*. 

By the optimality of x* we have that, w T ■ (x* — p(a)) > 0. Hence, w T ■ (x* — a) > 0. By 
the definition of a, we have 



~T 
W 



[x*-a)=w(S e )-w(S )+ (13) 

{u£S \d(v,u)=2t} 



We now deal with two cases: (i) {u E So \ d(v, u) = 2t] = 0, namely So does not contain 
a leaf of Gwjw, and (ii) {u E So \ d(v, u) = 2t} ^ 0, namely So contains at least one leaf 

Of GB(v,2t)- 

If {u E S I d(v, u) = 2t} = 0, then we conclude that w{S £ ) - w(S ) > 0. 

If {u E So I d(v, u) = 2i] 7^ 0, then following Equation (fT3l) it suffices to show that 
w T ■ (x* - a) - Y.{ues \d{v,u)=2t} ^w(!) ^ °- Let a = n (®) denote the projection of the 
assignment a. It holds that 



w T ■ (x* - a) w T ■ (x* - a) 



I J.' i-X ||-^ II 1 1 



(14) 



We therefore have by Definition [T71 that 

w T -(x* -a) >c(V,w)- p*-a*|| x . (15) 

Notice that \\x* — a*\\ x = S\{uE S a \ d(v,u) = 2t] , namely \\x* — a*\\ l equals to the 
number of variable vertices in the skinny tree Ts that are not leaves of GB(v,2t)- Let i = 

{u E S I d(v, u) = 2t] denote the number of leaves of GB(v,2t) that are contained in S. By 

equations (fT3l) and (TTBI) we have 

- w(S ) > c(V, w)-(\§\ -£)-£■ w max . (16) 

Therefore, it suffices to show that the number of variable vertices in the skinny tree Ts that are 
not leaves of Gbodm is greater or equal than the sum of the unit variable functions 4>a(l) of the 
leaves u of Gsiv,2t) that are contained in S. 

Indeed, if the maximum degree of a variable vertex in G is two, then a skinny tree is sim- 
ply a path for which £ E {1,2}. In this case we conclude with a proof idea of Sanghavi et 
ol. HSMWlll Proof of Lemma 3, Case 2] as follows. Because the root v is in Ts it holds that 



5* 



> £ ■ t. Hence, for t > + 1 it holds that c{V, w) ■ ( 

concludes the proof of the claim. □ 



S 



— £ ■ w max > which 
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